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Background / Context: 

Description of prior research and its intellectual context. 

Students entering high school in 9* grade face a formidable challenge. The transition to high 
school from 8* grade brings with it increased risks for all students. For example, students in 9th 
grade are anywhere from three to five times more likely to fail a class than students in any other 
grade (Southern Regional Educational Board, 2002). Similarly, ninth grade retention rates are 
higher than in any other grade (Smith, 2006). More importantly, research indicates that 70 to 80 
percent of students failing in 9* grade will eventually dropout of high school (Wyner, 2007). 

Early College High Schools (ECHS) are a new model merging high school and college designed 
to increase the number of students who graduate from high school and enroll and succeed in 
college. This pa^er presents results from a longitudinal, experimental study of ECHS, tracking 
students from 9“ grade through graduation from high school and enrollment in college. Results 
from earlier analyses have shown a variety of positive statistically significant impacts including: 

• ECHS students were more likely to be on-track for college than control students. 

• Compared to control students, ECHS students had better attendance, lower suspensions, 
and were more likely to remain in school. 

• ECHS students were more likely to report positive school experiences, including better 
relationships with staff, higher expectations, more rigorous and relevant instruction, and 
more frequent and varied support. 

Purpose / Objective / Research Question / Focus of Study: 

Description of the focus of the research. 


This paper will examine impacts of the ECHS model on “underprepared” students, defined as 
those students who did not pass the 8* grade state exams in either or both reading and math. 
“Prepared” students, on the other hand, are those passing both exams. In this paper, we report on 
the impact of the model on the performance gaps between underprepared and prepared students 
from 9* - 1 1'*’ grades. This paper addresses the following specific research questions: 

1 . Are underprepared students more likely to be from traditionally underrepresented groups 
such as first generation, minority and free/reduced price lunch students? 

2. What is the impact of the ECHS model on these students’ academic performance? 

3. How does that impact compare to the impact on students who are prepared? 

Setting: 

Description of the research location. 

This Study focuses on ECHSs as implemented in North Carolina. North Carolina has the largest 
concentration of ECHSs in the country, with over 70 across the state; all managed by the same 
entity, the North Carolina New Schools. The 19 schools that are a part of this study are 
geographically distributed throughout the state; they include schools in rural and urban areas, 
schools with predominantly white populations and schools with predominantly African- 
American populations. All of them are autonomous schools managed by the local school district 
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in partnership with a higher education partner, most frequently a community college. All 19 of 
the schools in the study are physically located on the campus of their higher education partner. 

Population / Participants / Subjects: 

Description of the participants in the study: who, how many, key features, or characteristics. 


In this paper, we include results from impact analyses completed to date on a longitudinal sample 
of 1,350 students who started high school in the 2005-2006, 2006-2007, 2007-2008, and 2008- 
2009 school years and have completed 11* grade. In addition, to address the first research 
question, we include data from a larger cross-sectional sample of all 9* graders for whom we 
have complete data (n=2,825). Table 1 presents a summary table of background characteristics 
for this 9* grade sample. The table shows that the sample is predominantly non-minority and 
female, with about half of the sample eligible for free or reduced price lunch and close to 40% 
having first generation college status. In addition, the table shows that the sample is equally 
distributed in background characteristics across the treatment and control groups, with the sole 
exception of a small statistically significant difference in free/reduced price lunch eligibility. 

-Please insert Table 1 here- 

Intervention / Program / Practice: 

Description of the intervention, program, or practice, including details of administration and duration. 

ECHSs are small, innovative high schools designed to increase the number of students who 
graduate from high school prepared for enrolling in postsecondary education. Frequently located 
on college campuses, early colleges serve students in grades 9-12/13. Students are expected to 
graduate within four to five years with a high school diploma and two years of transferable 
college credit. The target populations for these schools are students underrepresented in college, 
including students who are the first in their family to go to college and students who are low- 
income or members of a minority group underrepresented in college. As implemented in North 
Carolina, schools are expected to implement a specific set of design principles associated with a 
high quality school. Figure 1 presents a graphic representation of the conceptual framework of 
the ECHS model, connecting these design principles with the expected outcomes of the model. 

-Please insert Figure 1 here- 


Research Design: 

Description of the research design. 


This paper reports results from an lES -funded longitudinal experimental study of the impact and 
implementation of North Carolina’s ECHS model. Participating schools agreed to use a lottery to 
select students and the study is tracking outcomes for students randomly accepted into the 
program and those not accepted who enrolled somewhere else. Schools could enroll in the study 
at any point; as a result, our sample size increases each year as more schools provide students. 

To create the student sample, schools identified an eligible pool of applicants. The research team 
randomly ordered the list of students from lowest to highest and schools offered students spots in 
the order in which they appeared on the list. Students on the waitlist who were offered spots in 
the correct order were included in the treatment group. 
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Data Collection and Analysis: 

Description of the methods for collecting and analyzing data. 

We use data collected by the North Carolina Department of Public Instruction and housed at the 
North Carolina Education Research Data Center, which creates encrypted longitudinal data files 
by student. The outcomes looked at in this paper include academic course -taking and on-track 
progress in the subject areas of English, math, science and social studies, as well as overall 
progress. Eor this study, we developed an on-track indicator that identified the last possible year 
in which a student could take a course required for University of North Carolina system entrance 
without taking two courses in that subject area in the same year. Eor example, in 9^’’ grade, a 
student would have to pass at least one English course and one college preparatory math course. 
In lO^*' grade, students need to have passed at least two English courses, two math courses, and 
one science course. In 1 1* grade, they need to have passed three English courses, three math 
courses, two sciences and one social studies course. In addition, the study uses demographic 
data, including gender, race/ethnicity, free and reduced-price lunch status (E/R status), disability 
status, English Eanguage Eearner status, and parents’ educational level. 

We divided the sample into two sub-groups, according to whether students were identified as 
underprepared for 9**’ grade. Eor each outcome, we calculated unadjusted means for the treatment 
and control groups within each sub-group. We calculated impact estimates using multivariate 
linear regression models including site fixed effects, interaction of the treatment indicator with 
the site indicators, producing site level impact estimates, and baseline student characteristics 
including gender, race/ethnicity, E/R status age, and whether a student was retained prior to 8* 
grade. The site-specific impact estimates were then averaged proportionally to the number of 
students in each site to yield the overall impact estimate for each outcome. 

This study was guided by an intent-to-treat (ITT) analysis, which keeps all study participants in 
the group to which they were originally assigned, regardless of whether participants actually 
received the entire intervention or not. In this study, any students initially assigned to the early 
college were included in the treatment group, even if they changed their mind and did not go (no- 
shows). In addition, students initially identified as being in the control group remained there for 
anal}^ic purposes, even if they later attended the early college for any reason (crossovers). 

As the ITT approach may understate the ECHS effect on those who ended up participating in the 
intervention (Hollis & Campbell, 1999) . Thus, we also calculated the treatment-on-the-treated or 
local average treatment effect (EATE), which is calculated by dividing the ITT impact estimates 
by the factor (I-r-c) where r is the no-show rate and c is the crossover rate (Angrist et ah, 1996 ; 
Gennetian et ah, 2005) . In this study, we report the unadjusted means for each group — treatment 
and control — as well as adjusted impact estimates for both the ITT and EATE analyses. 

In addition to looking at the impact of the model on underprepared students, we also wanted to 
determine whether this impact was different from the impact on students who were prepared. In 
other words, we sought to understand if the program was increasing or decreasing the gap in 
performance between underprepared and prepared students. We conducted pair-wise 
comparisons between the estimated impacts for students who were underprepared vs. those who 
were prepared and tested whether these differences were statistically significant. 
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Findings / Results: 

Description of the main findings with specific details. 


To address the first research question, we conducted a series of analyses comparing the 
percentage of students prepared for 9* grade by the three primary subgroup indicators: 
free/reduced price lunch status, first generation college status and underrepresented minority 
status. We conducted these analyses overall and separately for each treatment group. As Table 1 
indicates, in all cases, the percentage of students prepared for 9* grade was statistically 
significantly lower among students categorized as free/reduced lunch, first generation college 
and underrepresented minority. For example, only 74% of students in the ECHS group who 
were on free/reduced price lunch were prepared for 9* grade compared to 87% of students not on 
free/reduced price lunch. From these results, we see that underprepared students for 9* grade are 
more likely to be on free/reduced price lunch, first generation college status and a member of an 
underrepresented minority across both treatment groups and overall. 

-Please insert Table 2 here- 
in order to address the 2"‘^ and research questions, we conducted a series of analyses on a 
cohort of 1,350 9* grade students who had progressed through 1 1* grade. Table 2 presents the 
results from these analyses for the core academic subject areas of English, math, science and 
social studies, as well as overall course-taking and progress. Across most outcomes, the impacts 
on underprepared students were greater compared to the impacts on prepared students. For 
example in 10* grade math, the impact on underprepared students progressing in that course was 
close to 24 percentage points (p<.05) compared to an impact among prepared students of 12 
percentage points, yielding a differential impact of 1 1 percentage points (p<.05). For science the 
impact on 1 1* grade underprepared student course-taking was statistically significantly larger 
compared to prepared students (p<.05). For English and social studies, however, none of the 
impacts on students as well as their differential impacts was found to be statistically significant. 
We thus see that the overall effect is driven primarily by math which appears to be a 
differentiating factor between the HS experience of underprepared and prepared students. 

-Please insert Table 3 here- 


Conclusions: 

Description of conclusions, recommendations, and limitations based on findings. 


Results from this experimental study show that it is possible for schools to have an impact on 
students entering high school at-risk for academic failure and dropout. These data suggest to us 
that the reduction or elimination of performance gaps in the early college is a product of a 
purposeful implementation of a high quality learning environment with high expectations, 
rigorous courses and instruction, positive relationships, extensive student support, and teachers 
taking responsibility for student learning. We intend to bolster these analyses with additional 
data on other outcomes such as attendance, suspensions, and aspirations for college. This 
analysis will represent one of the few studies conducted that combines both a rigorous study 
design and a focus on those students most at-risk. Most of the previous work that has been 
presented from this study has focused on students’ progression transition from high school to 
postsecondary study. This paper shifts the focus somewhat to look at the issue of ensuring 
successful transition from middle school to high school for academically at-risk students within 
the context of ensuring that all students stay on track for progressing successfully through high 
school. 
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Table 1: Background Sample Characteristics 



Whole 

Sample 

(N=2825) 

Treatment 

Group 

(N=1681) 

Control 

Group 

(N=1144) 

T-C Difference 


Mean 

Mean 

Mean 

Difference 

P-Value 

Race & Ethnicity 

American Indian 

0.7% 

0.8% 

0.6% 

0.2% 

0.47 

Asian 

1.1% 

1.1% 

1.1% 

0.0% 

0.80 

Black 

25.6% 

26.0% 

25.0% 

1.0% 

0.54 

Hispanic 

7.9% 

8.1% 

7.6% 

0.5% 

0.36 

Multi-racial 

3.6% 

3.5% 

3.8% 

-0.3% 

0.53 

White 

61.1% 

60.6% 

61.9% 

-1.4% 

0.32 

Gender 

Male 

39.7% 

39.8% 

39.5% 

0.3% 

0.55 

Exceptionality 

Disabled/Impaired 

1.7% 

1.6% 

1.8% 

-0.2% 

0.70 

Gifted 

11.0% 

10.2% 

12.0% 

-1.9% 

0.25 

Eirst Generation College 

39.3% 

38.5% 

40.5% 

-2.0% 

0.91 

Eree/Reduced Price Lunch Eligibility 

49.0% 

50.0% 

47.4% 

2.7% 

0.01* 

Retained 

2.5% 

2.2% 

3.0% 

-0.9% 

0.12 
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Figure 1: Logic Model for North Carolina’s Early College High Schools 


ECHS Design Principles 
College Ready 

Articulated program of study, grades 9-12 or 13, 
leading to Associate's degree or 2 yrs college 
credit 

College readiness activities 

Powerful Teaching and Learning: 

High-quality, rigorous, and relevant instruction 
Student collaboration and discussion 
Formative and multiple assessments 
Common standards 

Personalization 

Academic and affective supports 
Supportive relationships 

Professionalism: 

Ongoing professional development 

Collaboration among staff 

Collective responsibility and decisionmaking 

Purposeful Design: 

Autonomous governance 
Located on college campus 
Small size 
Flexible use of time 
Integration with college 


Intermediate Outcomes 


Long Term Outcomes 


Increased student attendance 


Increased frequency of higher 
level courses 


Improved attitudes toward self 
and school 


Improved behavior 


Increased aspirations toward 
college 


Increased graduation 
rates 


Increased enrollment in 
college 


Increased graduation from 
college 


Improved student 
achievement 
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Table 2. Percent Students Prepared for 9**^ Grade by Demographic Characteristics 


Prepared for 9“* 
Grade 

Free/reduced Price 
Lunch 

First Generation 
College 

U nderrepresented 
Minority 

No 

Yes 

Difference 

No 

Yes 

Difference 

No 

Yes 

Difference 

ECHS (n=l,681) 

87 % 

74 % 

14 % * 

84 % 

75 % 

9 %* 

87 % 

67 % 

20 % * 

Control (n=l,144) 

86 % 

64 % 

22 % * 

81 % 

69 % 

12 %* 

84 % 

59 % 

25 %* 

Total (n=2,825) 

87 % 

70 % 

17 %* 

83 % 

72 % 

10 %* 

86 % 

64 % 

22 % * 


* Denotes statistically significant differences at the p<0.05 level 
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Table 3. Student Outcomes: Students Who Passed vs. Did Not Pass Both Eighth Grade 


Reading and Math Who Took or Progressed in Enough Courses to be On-Track for College 


Outcomes 

Did Not Pass Both Eighth Grade 
Reading and Math 

Passed Both Eighth Grade Reading 
and Math 

Did Not Pass - 
Passed 

Unadjusted Means 

Adjusted 

Impacts 

Unadjusted Means 

Adjusted 

Impacts 

ECUS 

(N=231) 

Control 

(N=178) 

ITT 

(A) 

LATE 

ECUS 

(N=563) 

Control 

(N=378) 

ITT 

(B) 

LATE 

Differential 
Impact (A-B) 

1 Overall 

9* Grade 

% Take-Up 

88.9 

77.0 

11.5* 

12.5 

99.8 

93.8 

5.2* 

5.6 

6.3 

% Progress 

78.6 

66.3 

8.6* 

9.4 

97.9 

89.9 

7.3* 

7.9 

1.3 

10* 

Grade 

% Take-Up 

75.1 

52.2 

20.6* 

22.5 

98.2 

87.4 

10.0* 

10.8 

10.6* 

% Progress 

68.4 

43.2 

24.0* 

26.1 

95.4 

83.3 

12.6* 

13.6 

11.4 

11* 

Grade 

% Take-Up 

73.3 

50.1 

22.0* 

23.9 

94.8 

85.4 

10.6* 

11.5 

11.3* 

% Progress 

62.9 

44.6 

14.7* 

16.0 

91.3 

81.4 

10.4* 

11.2 

4.3 

English 

9* Grade 

% Take-Up 

99.6 

97.8 

1.8'^ 

2.0 

99.8 

100.0 

-0.2^^ 

-0.2 

2.0 

% Progress 

92.0 

93.8 

-3.3 

-3.6 

98.6 

98.4 

0.2^ 

0.2 

-3.5 

10* 

Grade 

% Take-Up 

96.4 

93.2 

2.4 

2.6 

99.5 

99.1 

0.4^ 

0.4 

2.1 

% Progress 

90.2 

84.2 

5.1 

5.5 

98.2 

96.4 

2.0 

2.1 

3.1 

11* 

Grade 

% Take-Up 

92.9 

88.3 

3.2 

3.5 

98.0 

96.9 

1.8 

1.9 

1.5 

% Progress 

86.2 

83.8 

-0.6 

-0.6 

95.5 

93.8 

1.6 

1.7 

-2.2 

Math 

9* Grade 

% Take-Up 

88.9 

77.0 

11.5* 

12.5 

100.0 

93.8 

5.3* 

5.8 

6.1 

% Progress 

83.5 

67.4 

12.9* 

14.1 

98.4 

89.9 

7.8* 

8.4 

5.1 

10* 

Grade 

% Take-Up 

75.1 

52.2 

20.6* 

22.5 

98.2 

87.4 

10.0* 

10.8 

10.6* 

% Progress 

69.7 

44.3 

23.5* 

25.6 

95.9 

84.1 

12.1* 

13.0 

11.5* 

11* 

Grade 

% Take-Up 

75.1 

51.3 

22.8* 

24.9 

95.8 

86.0 

10.9* 

11.7 

12.0* 

% Progress 

66.5 

46.8 

16.7* 

18.1 

93.2 

82.7 

10.8* 

11.6 

5.9 

Science 

10* 

Grade 

% Take-Up 

99.6 

97.2 

2.4^ 

2.6 

100.0 

99.7 

0.3^^ 

0.3 

2.1 

% Progress 

97.8 

93.2 

3.9 

4.2 

99.3 

99.0 

0.3^ 

0.3 

3.6 

11* 

Grade 

% Take-Up 

98.2 

92.7 

4.3* 

4.7 

99.7 

99.5 

O.U 

0.2 

4.2* 

% Progress 

93.8 

88.2 

4.5 

4.9 

99.1 

98.6 

0.6^ 

0.6 

3.9 

Social Studies 

11* 

Grade 

% Take-Up 

99.6 

98.3 

1.2^ 

1.4 

99.8 

100.0 

-0.2^^ 

-0.2 

1.4 

% Progress 

97.3 

96.1 

1.0 

1.1 

99.5 

99.3 

0.2^^ 

0.2 

0.8 


* Denotes statistically significant differences at the p<0.05 level. 

Because the means for these outcomes are close to 100%, the ITT estimate is set to be equal to the unadjusted mean, 
and the p-value is calculated using Fisher's exact test. 
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